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Abstract 

This paper consider the problem of determining the rehabihty of a software 
system which can be decomposed in a number of modules. We have derived the 
expression of the reliability of a system using the Markovian model for the transfer 
of control between modules in order. We have given the expression of reliability 
by considering both benign and catastrophic failure. The expression of reliability 
presented in this work is applicable for some control software which are designed 
to detect its own internal errors. 



1 Introduction 

Now a days large scale software systems are used in every walk of life. The price of 
software are much higher than the cost of hardware when we consider a huge computer 
intensive system. Moreover the penalty cost incurred by a false outcome of a system 
is enormous. To address such a challenge posed by this technological trend, during the 
last three decades extensive research has focused on the area of software reliability. The 
consideration of software reliability is increasing because of the growing emphasis on 
software that is reusable (as opposed to software that is written for a terminal mission), 
where it is essential to demonstrate that the system will perform reliably for a variety of 
end-user applications. 

A software system is defined here as a " collection of programs and system files such 
that the system files are accessed and altered only by the programs in the collection ". 
Each element in this collection will be called a module - for instance, a module might be 
a program, a subprogram, or a file. The performance ( and hence the reliability ) of the 
system clearly depends on that of each individual module and the relationship between 
these modules and the system; in this regard a software system is quite similar to any 
other system. However, the actual relationship between system and module reliabilities 
is quite unique and depends on the specific definition of software reliability as well as on 
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the structure of the overall system. In this paper we focus on software systems that can 
be decomposed into a finite number of modules. 

In testing a software one may test the system as a whole, but in practice, different 
organizational entities are assigned responsibility of developing different modules. So it 
will be more beneficial in the context of both cost and time test the individual modules 
instead of testing them together. In order to do this, some mathematical models, of- 
ten referred to as Software Reliability Growth Models (SRGM) are used to enable the 
software reliability practitioners to estimate the expected future reliability of a software 
under development and accordingly allocate time, money, human resources to a project. 
Often these reliability growth models express software system reliability in terms of the 
individual module reliabilities which is favorable under both time and cost constraints. 

Cheung (2), first expressed the system reliability in terms of the component relia- 
bilities. Poore et al. (1) suggest allocating the targeted system reliability goal among 
the components and then testing the individual components to verify whether the com- 
ponent reliabilities meet the allocated goals at a specified level of confidence, where as 
Easterling, Mazumdar, Spencer and Diegert (6), has discussed this method may lead to 
estimates of overly conservative sample size requirements for component testing. Yang 
et. al. has implemented the idea of using testability to estimate software reliability. 
They have also provided the basic steps involve estimating testability, evaluating how 
well software was written, and assessing the relationship between testing and usage by 
assuming the modules are independently functioning. They have also compared their 
results with those obtained by using two reliability growth models. Rajgopal et. al. 
has used a Markovian model for the transfer of control between modules in order to 
develop the system reliability expression in terms of the module reliabilities in case of a 
dependent setup. They have also discussed a procedure for determining the minimum 
number of tests required of each module such that the probability of certifying a system 
whose reliability falls below a specified value Rq is less than a specified small fraction f3. 
Bondavalli et. al. has considered the concept of benign failure and catastrophic failure 
for determining the software reliability for a iterative program. 

In this paper we have expressed the system reliability in terms of testability of a 
particular module following Yang et. al. for dependent modules and have introduced 
the concepts of benign and catastrophic failure following Bondavalli et. al. in case of a 
system where it can be decomposed in a finite number of dependently functional mod- 
ules. The section 2 discuss the notations and preliminaries, section 3 gives the expression 
of the probability of correct output for a specific input. Recent research [26] has shown 
a strong correlation between reliability and coverage criteria (Lott et al. (2005), Khun 
et. al. (2002), Yilmaz et. al. (2004) etc.), although it is very difficult to quantify this 
relation. Dalai et al. [6] and many more has examined this relationship between unit-test 
statement coverage and system-test faults later attributed to those units. 

Present work has been organized in 4 sections the section 2 gives the notation and 
preliminaries of software reliability in terms testability of a module. In the 3rd sections 
we have derived the probability of correct output of a particular system corresponding 
to a particular input considering both the case presence and absence of benign failure. 
In Section 4 we present a brief discussions about the procedure mentioned here. 
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2 Notations and Preliminaries 



There is no rigorous definition of 'Quality'. But it can be weakly defined as the fit- 
ness of purpose of any product to its users. Similarly software quality is defined as 
the conformance to explicitly stated functions and performance requirements, explicitly 
documented development standards and implicit characteristics that are expected of all 
professionally crafted software(Cai Kai-Yuan Cai (3)). Alternatively, the quality of a 
software may be characterized by some quality factors of a software - reliability, effi- 
ciency, correctness, usability, testability etc. 

Reliability of a software system may be viewed as the expected value of probability 
of failure-free operation of a program for a randomly chosen set of input variables. The 
term failure in the context of software reliability implies a result other than what was 
expected from the software for a set of inputs. Following Voas et. al. (1995) we define 
the testabihty of a particular system as the probability of failure of the system for a par- 
ticular input when it is assumed that there is at least one fault in the system. Suppose 
we have a software system which can be decomposed in N modules. Thus the testability 
of a particular module, say ith (Vi — l(l)A^) module, is given by 

Pi — Prob[ that the ith module will give incorrect output | there is at least one fault, 
probability distribution of input] (1) 

The expression for the probability that the ith module will contain error if the module 
has tested rii times successfully, is given by the following (Yang et. al. (1998)) 

where Q;j(0) is the probability of failure of the system before testing. Let nt{x) is the 
probability of a system giving correct output corresponding to a particular set of input 
X. The expression of 7Tt{x) by assuming the independent setup is given by (Yang et. al. 
(1998)) 

TTtix) = Y[{1 - Qiaiit)) (3) 

where qi is the revealibility of the i th module and S{x) is the set of those modules which 
will be executed by the input x. The reliability of a software system is given by 

Rt^ T:t{x)^{x)dx (4) 

JxeX 

where X is the set of all possible inputs and (f){x) is the probability distribution of x. 



3 Detailed Expression of Tit{x) for Dependent Setup 

A software system is necessarily an iterative. In each iteration a particular module 
accepts a value and produce an output. The outcomes of an individual iteration may 
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be: i) success, i.e., the delivery of a correct result, ii) a benign failure of the program, 
i.e., an output that is not correct but does not, by itself, cause the entire mission of 
the controlled system to fail, or iii) a catastrophic failure, i.e., an output that causes 
the immediate failure of the entire mission. The characterization of failures in benign 
and catastrophic is discussed with example by Bondavalli. et. al. (). In this section 
we derive the expression of 7rt{x) first of all only considering the catastrophic failure 
and then in the subsequent subsection considering the benign and catastrophic failure 
simultaneously. 



3.1 Expression of 7rt{x): No Benign Failure in the System 



Consider the above software system with N modules. Let pij be the probability that the 
control from the ith module will be transferred to the jth module with correct execution 
(Vi = 1(1)A^, Vj = l(l)A^). Let be a state of successful completion of the system. 
As S is achievable from any one of the module so we define pis (Vi = 1(1)-^) as the 
probability of successful completion of the mission from the ith module. Here we must 
have Pis + E]=iPtj = 1- 

As we have a faulty system, that is, we have a system where there is at least one 
fault or if the faults can be classified into categories then there are at most one fault 
of each category. So we introduce another state F, i.e., unsuccessful completion of the 
mission. As any module may be faulty so the state F also can be achieved from any of 
the module. We define pip as the probability of unsuccessful completion of the module i 
(Vi = 1(1)A''). The transition probability matrix takes the following form for the above 
setup. 



/ pn(l-«?(t)) Pi2(l-«?W) 
P2i(l-af(t)) P22(l-ai(t)) 



Q 








PiN{l-o.f{t)) Pisil-aUt)) af{t) \ 
P2iv(l-af(t)) P25(l-«i(t)) afW 



1 

1 



(5) 



where is the probability of faulty completion of the iih. module for the input x. 

The expression of a!f{t) is given by 



a'lit) = qiai{t) 



(6) 



If we assume that the first block is the control block then the probability of correct 
completion of the mission for the given input x is given by (Parzen (1962)) 



N 



7r,(x) = Y.ilN - Q)^lPis{l - <(i)) 



(7) 



where Q is the sub-matrix of Q deleting its last two columns and rows. 
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3.2 Expression of 7rt{x): Benign Failure and Catastrophic Fail- 
ure are in the System 

Prom the software viewpoint solely, and without referring to any specific application, we 
assume here that all detected failures (default safe values of the control outputs from the 
computer) do not prevent the mission to continue and are in this sense benign, whereas 
undetected failures are conservatively assumed to have a "catastrophic" effect on the 
controlled system. Obviously, if knowledge of the consequences of software failures on 
the system was available for a specific system, the proper splitting of software failures 
into benign and catastrophic could be precisely made. We make the following assumption 
to model the system. 

Suppose 5* 5* is a state where the total system, that is all the N modules, runs without 
any fault of either kind. Let be the state where the system is running in benign failure 
of ith level, that is after i iterations the system will enter in the state SS. As the previous 
subsection S and F denotes the successful completion of the mission and completion of 
the mission with a failure respectively. The mission will fail if their is a catastrophic 
failure in the system. Let us also assume that if there is a benign failure of length greater 
than a threshold value, say Uc, then the system will enter in a catastrophic failure region. 
Although this assumption will take the model a little away from reahty, a model should 
be good enough to handle a benign failure of any arbitrary random length, but this 
assumption will make the calculation of reliability expression easier which will increase 
its practical application. At this point note that the state S, that is the successful 
completion of the program, can be achieved only from the state SS, where as the state 
F can be achieved from any of the state SS or BiS (Vi = l(l)A^), but we assume here 
the control will be transferred from the state Bi to only to reduce the number of 
parameters in the model. 

The transition probability matrix will be as follows 
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(8) 



Here the matrix Qoo is a x matrix which describes that the flow is running without 
entering in benign failure or catastrophic failure. The matrix Qq^ is also a x A^ matrix 
giving the transition probabilities of the flow of control from stable state to the fcth level 
benign failure (VA; = l(l)nc). Similarly, the matrix which is also N x N denotes the 
transition probabilities of the control entering from the kth level benign failure to I th 
level (V/c = l(l)ncV/ = l(l)nc). From the A;th level benign failure we can only achieve 
the k — 1th level benign failure so Q\.i = O (V/ 7^ A; — 1). Where O is the null matrix of 
order N x N. 5*° is a A" x 1 vector of the transition probabilities of successful completion 
of the mission from the stable state. As the mission can terminate successfully only from 
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the stable state so the rest of the entries in this column are all zero. denotes a null 
vector of length N and 0' denotes transpose of 0. Finally, is a column vector of length 
N giving probabilities of reaching the state of catastrophic failure from the stable state. 

To give the structure of sub- matrices Qoo, let us define pff be the probability of the 
control to enter from the ith module to jth module in the state SS. So the matrix Qoo 
is given by 



Q 



00 



/ Pn 



pfi 
pif pH 



V Pm 



Pn% 



Pin \ 

P2N 



... pffr, J 



(9) 



Let us also define pfj^ be the probability that the control will be transferred from the 
module i to the module j from the state SS to any of benign failure. Let also the 
probability that the control will enter in B^, thus the probability that the control will 
enter in the jth module from the ith module in the state is given by pf^p^- So the 
matrix will take the following form 



Q^Ok 



( PllPk PfiPk 
PifPk PliPk 



KPmPk PmPk 



PmPk \ 

pIM 

PNNPk J 



(10) 



If pis and pip is respectively the successful completion of the mission and achieving 
catastrophic failure from the ith module. Then we must have 



N Uc N 

Y.Pl' + Y.PkY. Pff + P^S + P^F 

j=l k=l j=l 



1 Vi = 1{1)N 



The matrix Qlk-i takes the following form 



Here we have 
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(12) 



(13) 



Finally, the matrix Q\q is the matrix of transition probabilities, say p^^, that the flow of 
control will be transferred from the ith to the jth module and from the B^ to SS. Here 
also 



N 



Y,p\^ = 1 Vi = l{l)N 



(14) 
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By assuming as before the first module as tfie control module the expression of Trt{x) is 
given 

N 

M^) = Y^i^Nuc - Q)uPiS (15) 
i=l 

where Q is once again the sub-matrix of Q deleting its last two columns and rows. 

4 Conclusions 

In this work we have given an expression of the reliability of a software system which can 
be divided in a finite number of modules. The transition probabilities we have considered 
can be easily estimated using maximum likelihood method of estimation. 

Consider the setup without benign failure, suppose ith block is tested times, out of 
which x^j times the control is transferred to the jth state (Vi = l(l)A^&Vj = l(l)iV, S, F). 
The maximum likelihood estimates oipij is a;*/ (Z^^i a;* +Xg) and that of af (t) is Xp/rii. 
Hence estimate of nt{x) can be obtained and let it be denoted by ^t(a;). Finally the 
estimate of reliability of a system can be given by 

^* = ^ E ^t(^) (16) 

where W is the set of all inputs which are used for testing. This is an extension of some 
previous work and the model what we have considered are more realistic for some control 
software which are designed to detect its own internal errors and then issue a safe output 
and reset itself to a known state from which the program is likely to proceed correctly. 
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